At the and of 2016 I had the pleasure to attend the 11th Latin American Conference on Pattern Languages of Programs, a.k.a SugarLoaf PLoP. PLoP is a series of conferences on Patterns (as in Design Patterns ), a subject that I appreciate a lot. Each of the PLoP conferences but the original main big conference has a funny name. SugarLoaf PLoP is called that way because its very first edition was held in Rio de Janeiro, so the organizers named it after a very famous mountain in Rio. The name stuck even though a long time has passed since it was held in Rio for the last time. 2016 was actually the first time SugarLoaf PLoP was held outside of Brazil, finally justifying the Latin American part of its name. I was presenting a paper I wrote on patterns for testing Debian packages. The Debian project funded my travel expenses through the generous donations of its supporters. PLoP s are very fun conferences with a relaxed atmosphere, and is amazing how many smart (and interesting!) people gather together for them. My paper is titled Patterns for Writing As-Installed Tests for Debian Packages , and has the following abstract:

Large software ecosystems, such as GNU/Linux distributions, demand a large amount of effort to make sure all of its components work correctly invidually, and also integrate correctly with each other to form a coherent system. Automated Quality Assurance techniques can prevent issues from reaching end users. This paper presents a pattern language originated in the Debian project for automated software testing in production-like environments. Such environments are closer in similarity to the environment where software will be actually deployed and used, as opposed to the development environment under which developers and regular Continuous Integration mechanisms usually test software products. The pattern language covers the handling of issues arising from the difference between development and production-like environments, as well as solutions for writing new, exclusive tests for as-installed functional tests. Even though the patterns are documented here in the context of the Debian project, they can also be generalized to other contexts.

In practical terms, the paper documents a set of patterns I have noticed in the last few years, when I have been pushing the Debian Continous Integration project. It should be an interesting read for people interested in the testing of Debian packages in their installed form, as done with autopkgtest. It should also be useful for people from other distributions interested in the subject, as the issues are not really Debian-specific. I have recently finished the final version of the paper, which should be published in the ACM Digital Library at any point now. You can download a copy of the paper in PDF. Source is also available, if you are into markdown, LaTeX, makefiles and this sort of thing. If everything goes according to plan, I should be presenting a talk on this at the next Debconf in Montreal.

My parents were cleaning out a bunch of books they didn't want, so I grabbed some of the ones that looked interesting. A rather wide variety of random stuff. Also, a few more snap purchases on the Kindle even though I've not been actually finishing books recently. (I do have two finished and waiting for me to write reviews, at least.) Who knows when, if ever, I'll read these. Mark Ames Going Postal (nonfiction)
Catherine Asaro The Misted Cliffs (sff)
Ambrose Bierce The Complete Short Stores of Ambrose Bierce (collection)
E. William Brown Perilous Waif (sff)
Joseph Campbell A Hero with a Thousand Faces (nonfiction)
Jacqueline Carey Miranda and Caliban (sff)
Noam Chomsky 9-11 (nonfiction)
Noam Chomsky The Common Good (nonfiction)
Robert X. Cringely Accidental Empires (nonfiction)
Neil Gaiman American Gods (sff)
Neil Gaiman Norse Mythology (sff)
Stephen Gillet World Building (nonfiction)
Donald Harstad Eleven Days (mystery)
Donald Harstad Known Dead (mystery)
Donald Harstad The Big Thaw (mystery)
James Hilton Lost Horizon (mainstream)
Spencer Johnson The Precious Present (nonfiction)
Michael Lerner The Politics of Meaning (nonfiction)
C.S. Lewis The Joyful Christian (nonfiction)
Grigori Medredev The Truth about Chernobyl (nonfiction)
Tom Nadeu Seven Lean Years (nonfiction)
Barak Obama The Audacity of Hope (nonfiction)
Ed Regis Great Mambo Chicken and the Transhuman Condition (nonfiction)
Fred Saberhagen Berserker: Blue Death (sff)
Al Sarrantonio (ed.) Redshift (sff anthology)
John Scalzi Fuzzy Nation (sff)
John Scalzi The End of All Things (sff)
Kristine Smith Rules of Conflict (sff)
Henry David Thoreau Civil Disobedience and Other Essays (nonfiction)
Alan W. Watts The Book (nonfiction)
Peter Whybrow A Mood Apart (nonfiction) I've already read (and reviewed) American Gods, but didn't own a copy of it, and that seemed like a good book to have a copy of. The Carey and Brown were snap purchases, and I picked up a couple more Scalzi books in a recent sale.

This weekend, I traveled to Columbus, Ohio to attend Ohio Linux Fest. I departed San Francisco early on Thursday. It was interesting getting to experience the luxurious side of flying as I enjoyed a mimosa in the American Express Centurion lounge for the first time. I even happend to cross paths with Corey Quinn, who was on his way to [DevOpsDays Boise]. While connecting in Houston, I met up with the always awesome Jos Antonio Rey, who was to be my travel companion for this trip. The long day of travel took its toll on us, so we had a lazy Friday morning before checking in for the conference around lunch time. I was not that interested in the afternoon sessions, so I spent the majority of the first day helping out at the Ubuntu booth and catching up with friends and colleagues. The day ended with a nice Happy Hour sponsored by Oracle. Saturday was the main day for the conference. Ethan Galstad, Founder and CEO of Nagios, started the day with a Keynote about Becoming the Next Tech Entrepreneur. Next up was Elizabeth K. Joseph with A Tour of OpenStack Deployment Scenarios. While I ve read plenty about OpenStack, I ve never actually used it before. As a result, this demo and introduction was great to watch. It was entertaining to watch her login to CirrOS with the default password of cubswin:), as the Chicago Cubs are currently playing the San Francisco Giants in the National League Divisional Series (and winning). Unfortunately, I was not able to win a copy of her new Common OpenStack Deployments book, but it was great getting to watch her signing copies for other attendees after all of the hard work that went into writing the book. For lunch, Jos , Elizabeth, and Svetlana Belkin all gathered together for an informal Ubuntu lunch. Finally, it was time for me to give my talk. This was the same talk I gave at FOSSCON, but this time, I had a significantly larger audience. Practice definitely makes perfect, as my delivery was a lot better the second time giving this talk. Afterwards, I had a number of people come up to me to let me know that they really enjoyed the presentation. Pro Tip: If you ever attend a talk, the speaker will really appreciate any feedback you send their way. Even if it is a simple, Thank You , it really means a lot. One of the people who came up to me after the talk was Unit193. We have known each other through Ubuntu for years, but there has never been an opportunity to meet in person. I am proud to be able to say with 99% confidence that he is not a robot, and is in fact a real person. Next up was a lesson about the /proc filesystem. While I ve explored it a bit on my own before, I still learned a few tips and tricks about information that can be gained from the files in this magical directory. Following this was a talk about Leading When You re Not the Boss. It was even partially taught by a dummy (the speaker was a ventriloquist). The last regular talk of the day was one of the more interesting ones I attended. It was a talk by Patrick Shuff from Facebook about how they have built a load balancer than can handle a billion users. The slide deck was well-made with very clear diagrams. The speaker was also very knowledgeable and dealt with the plethora of questions he received. Prior to the closing keynote was a series of lightning talks. These served as a great means to get people laughing after a long day of talks. The closing keynote was given by father and daughter Joe and Lilly Born about The Democratization of Invention. Both of them had very interesting stories, and Lily was quite impressive given her age. We skipped the Nagios After Party in favor of a more casual pizza dinner. Overall, it was a great conference, and I am very glad to have had the opportunity to attend. A big thanks to Canonical and the Ubuntu Community for fudning my travel through the Ubuntu Community Fund and to the Ohio Linux Fest staff for allowing me the opportunity to speak at such a great conference.

This post is long past due, but I figured it is better late than never. At the start of the year, I set a goal to get more involved with attending and speaking at conferences. Through work, I was able to attend the Southern California Linux Expo (SCALE) in Pasadena, CA in January. I also got to give a talk at O'Relly's Open Source Convention (OSCON) in Austin, TX in May. However, I really wanted to give a talk about my experience contributing in the Ubuntu community. Jos Antonio Rey encouraged me to submit the talk to FOSSCON. While I've been aware of FOSSCON for years thanks to my involvement with the freenode IRC network (which has had a reference to FOSSCON in the /motd for years), I had never actually attended it before. I also wasn't quite sure how I would handle traveling from San Francisco, CA to Philadelphia, PA. Regardless, I decided to go ahead and apply. Fast forward a few weeks, and imagine my surprise when I woke up to an email saying that my talk proposal was accepted. People were actually interested in me and what I had to say. I immediately began researching flights. While they weren't crazy expensive, they were still more money than I was comfortable spending. Luckily, Jos had a solution to this problem as well; he suggested applying for funding through the Ubuntu Community Donations fund. While I've been an Ubuntu Member for over 8 years, I've never used this resource before. However, I was happy when I received a very quick approval. The conference itself was smaller than I was expecting. However, it was packed with lots of friendly and familiar faces of people I've interacted with online and in person over the years at various Open Source events. I started off the day by learning from Jos how to use Juju to quickly setup applications in the cloud. While Juju has definitely come a long way over the last couple of years, and it appears t be quite easy to learn and use, it still appears to be lacking some of the features needed to take full control over how the underlying applications interact with each other. However, I look forward to continuing to watch it grow and mature. Net up, we had a lunch break. There was no catered lunch at this conference, so we decided to get some cheesesteak at Abner's (is any trip to Philadelphia complete without cheesesteak?). Following lunch, I took some time to make a few last minute changes to my presentation and rehearse a bit. Finally, it was time. I got up in front of the audience and gave my presentation. Overall, I was quite pleased. It was not perfect, but for the first time giving the talk, I thought it went pretty well. I will work hard to make it even better for next tme. Following my talk was a series of brief lightning talks prior to the closing keynote. Another long time friend of mine, Elizabeth Krumbach Joseph, was giving the keynote about listening to the needs of your global open source community. While I have seen her speak on several other occassions, I really enjoyed this particular talk. It was full of great examples and anecdotes that were easy for the audience to relate to and start applying to their own communities. After the conference, a few of us went off and played tourist, paying the Liberty Bell a visit before concluding our trip in Philadelpha. Overall, I had a great time as FOSSCON. It was great being re-united with so many friends. A big thank you to Jos for his constant support and encouragement and to Canonical and the Ubuntu Community for helping to make it possible for me to attend this conference. Finally, thanks to the terrific FOSSCON staff for volunteering so much time to put on this great event.

What happened in the Reproducible Builds effort between Sunday September 25 and Saturday October 1 2016: Statistics For the first time, we reached 91% reproducible packages in our testing framework on testing/amd64 using a determistic build path. (This is what we recommend to make packages in Stretch reproducible.) For unstable/amd64, where we additionally test for reproducibility across different build paths we are at almost 76% again. IRC meetings We have a poll to set a time for a new regular IRC meeting. If you would like to attend, please input your available times and we will try to accommodate for you. There was a trial IRC meeting on Friday, 2016-09-31 1800 UTC. Unfortunately, we did not activate meetbot. Despite this participants consider the meeting a success as several topics where discussed (eg changes to IRC notifications of tests.r-b.o) and the meeting stayed within one our length. Upcoming events Reproduce and Verify Filesystems - Vincent Batts, Red Hat - Berlin (Germany), 5th October, 14:30 - 15:20 @ LinuxCon + ContainerCon Europe 2016. From Reproducible Debian builds to Reproducible OpenWrt, LEDE & coreboot - Holger "h01ger" Levsen and Alexander "lynxis" Couzens - Berlin (Germany), 13th October, 11:00 - 11:25 @ OpenWrt Summit 2016. Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle (USA), November 11th-12th, 2016. Previous events GHC Determinism - Bartosz Nitka, Facebook - Nara (Japan), 24th September, ICPF 2016. Toolchain development and fixes Michael Meskes uploaded bsdmainutils/9.0.11 to unstable with a fix for #830259 based on Reiner Herrmann's patch. This fixed locale_dependent_symbol_order_by_lorder issue in the affected packages (freebsd-libs, mmh). devscripts/2.16.8 was uploaded to unstable. It includes a debrepro script by Antonio Terceiro which is similar in purpose to reprotest but more lightweight; specific to Debian packages and without support for virtual servers or configurable variations. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible in our testing framework after being fixed:

• ara/1.0.32 by Chris Lamb, original patch by Chris Lamb.
• fracplanet/0.4.0-5 by Chris Lamb, original patch by Reiner Herrman.
• gnarwl/3.6.dfsg-8 by Bernhard Schmidt, original patch by Chris Lamb.
• kgb-bot/1.34-1 by gregor herrmann, original patch by gregor herrmann.
• survex/1.2.29-1 by Olly Betts.
• zpaq/1.10-3 by Chris Lamb, original patch by Reiner Herrman.
• fig2dev/1:3.2.6-3 by Roland Rosenfeld.
• luxio/11-1 by Daniel Silverstone.
• monkeysign/2.1.1 by Antoine Beaupr , original patch by Daniel Kahn Gillmor.
• openarena-085-data/0.8.5split-9 by Simon McVittie.
• openarena-088-data/0.8.8-7 by Simon McVittie.
• openarena-data/0.8.5split-9 by Simon McVittie.
• rc/1.7.4-1 by Reiner Herrmann, original patch by Chris Lamb.

The following updated packages appear to be reproducible now for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• gkrellm/2.3.8-1 by Sandro Tosi
• glassfish/1:2.1.1-b31g+dfsg1-4 by Emmanuel Bourg

Some uploads have addressed some reproducibility issues, but not all of them:

• apache2/2.4.23-5 by Rapha l Hertzog
• freeradius/3.0.11+dfsg-1 by Michael Stapelberg
• libnss-ldap/265-4 by Chris Lamb
• lift/2.5.0-1 by Nicolas Delvaux
• linux/4.8~rc8-1~exp1 by Ben Hutchings
• nose2/0.6.5-2 by Barry Warsaw
• postgresql-9.6/9.6.0-1 Christoph Berg
• strace/4.13-0.1 by Nicolas Braud-Santoni
• yersinia/0.7.3-3 by No l K the

Patches submitted that have not made their way to the archive yet:

• #838888 filed against dh-haskell by Chris Lamb.
• #838971 filed against slang2 by Chris Lamb.
• #839587 filed against sympa by Chris Lamb.
• #839181 filed against transmission-remote-gtk by Chris Lamb.
• #838829 filed against vala by Sebastian Reichel.
• #838970 filed against webkit2pdf by Chris Lamb.
• #831569 filed against websockets by Chris Lamb.
• #839347 filed against xml-core by Lucas Nussbaum.
• #839526 filed against xml-core by Adrian Bunk.

Reviews of unreproducible packages 77 package reviews have been added, 178 have been updated and 80 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been updated:

• Added clilibs_line_order, records_build_flags and hevea_captures_build_path.
• Removed locale_dependent_symbol_order_by_lorder, fixed in bsdmainutils/9.0.11.
• Updated diffoscope_runs_forever, captures_build_path.

Weekly QA work As part of reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (3)
• Chris Lamb (12)
• Lucas Nussbaum (3)
• Sebastian Reichel (1)

diffoscope development A new version of diffoscope 61 was uploaded to unstable by Chris Lamb. It included contributions from:

• Ximin Luo:
  • Improve the CLI --help text and add an --output-empty option.
• Chris Lamb:
  • Add a progress bar and show it if stdout is a TTY. You can read more about it here. It can also be read by higher-level programs via the --status-fd CLI option.
• Maria Glukhova:
  • Behaviour improvements in the case of OS-level errors.
• Mattia Rizzolo:
  • Testing and packaging improvements.

Post-release there were further contributions from:

• Chris Lamb:
  • Code architecture improvements.
• Maria Glukhova:
  • Testing improvements.

reprotest development A new version of reprotest 0.3.2 was uploaded to unstable by Ximin Luo. It included contributions from:

• Ximin Luo:
  • Add a --diffoscope-arg CLI option to pass extra args to diffoscope.

Post-release there were further contributions from:

• Chris Lamb:
  • Code quality improvements.

tests.reproducible-builds.org

• Hans-Christoph Steiner continued work on setting up reproducible tests for F-Droid.
• Holger cleaned up the script creating the page showing breakages, so that it now also cleans up some of the breakage it finds.
• IRC notifications about diffoscope crashes and artifacts available for investigations have been dropped; instead the breakages page has a permanent pointer. (h01ger)
• IRC notifications from the automatic package scheduler and status changes for packages have been moved -- as a temporary trial -- to #debian-reproducible-changes on irc.oftc.net (Mattia).

Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

debci 1.4 was released just a few days ago. Among general improvements, I would like to highlight:

• pretty much every place in the web UI that mentions a PASS or a FAIL also displays the tested package version. This was suggested to me on IRC by Holger
• I also tried to workaround an instability when setting up the LXC containers used for the tests, where the test bed process setup would finish without failure even though some steps in the middle of it failed. This caused the very final step for the debci-specific setup to fail, so there was no debci user inside the container, which caused tests to fail because that user was missing. Before that was fixed I was always keeping an eye on this issue, fixing the issue by hand, and re-triggering the affected packages by hand, so as far I can tell there is no package whose status has been permanently affected by this.
• Last, but not least, this release brings an interesting contribution by Gordon Ball, which is keeping track of different failure states. debci will now let you know whether a currently failing package has always failed, has passed in a previous version, or if the same version that is currently failing has previously passed.

ci.debian.net has been upgraded to debci 1.4 just after that. At the same time I have also upgraded autodep8 and autopkgtest to their latest versions, available in jessie-backports. This means that it is now safe for Debian packages to assume the changes in autopkgtest 4.0 are available, in special the $AUTOPKGTEST_* environment variables. In other news, for several weeks there were had issues with tests not being scheduled when they should have. I was just assuming that the issue was due to the existing test scheduler, debci-batch, being broken. Today I was working on a new implementation that is going to be a lot faster, I started to hit a similar issue on my local tests, and finally realized what was wrong. The fact is that debci-batch stores the timestamp of the last time a package has been scheduled to run, and it there are no test result after that timestamp, it assumes the package is still in the queue to be tested, and does not schedule it again. It turns out that a few weeks ago, during maintainance work, I had cleared the queue, discarding all jobs that were there, but forgot to reset those timestamps, so when debci-batch came around again, it checked the timestamp of the last request and did not make new requests because there was no test result after that timestamp! I cleared all those timestamps, and the system should now go back to normal. That is it for now. I you want to contribute to the Debian CI project and want to get in touch, you can pop up on the #debci channel on the OFTC IRC network, or mail the autopkgtest-devel mailing list.

What happened in the Reproducible Builds effort between Sunday August 28 and Saturday September 3 2016: Media coverage Antonio Terceiro blogged about testing build reprodubility with debrepro . GSoC and Outreachy updates The next round is being planned now: see their page with a timeline and participating organizations listing. Maybe you want to participate this time? Then please reach out to us as soon as possible! Packages reviewed and fixed, and bugs filed The following packages have addressed reproducibility issues in other packages:

• ruby-ronn/0.7.3-5 by Antonio Terceiro, original patch by Chris Lamb.

The following updated packages have become reproducible in our current test setup after being fixed:

• check-all-the-things/2016.09.03 by Paul Wise, original patch by Chris Lamb.
• e2fsprogs/1.43.2-2 by Theodore Y. Ts'o.
• gnupg1/1.4.21-1 by Daniel Kahn Gillmor, #834755 by Chris Lamb.
• gnupg2/2.1.15-2 by Daniel Kahn Gillmor.
• inform6-compiler/6.33-2 by Ben Finney.
• libhat-trie/0.0~git25f9e946-2 by Sascha Steinbiss.
• mrd6/0.9.6-13 by Thomas Preud'homme, original patch by Reiner Herrmann.
• safecat/1.13-3 by Teemu Hukkanen, original patch by Reiner Herrmann.
• shibboleth-sp2/2.6.0+dfsg1-1 by Ferenc W gner.
• sweethome3d-furniture-editor/1.15-2 by Markus Koschany, original patch by Reiner Herrmann.
• traceroute/1:2.1.0-2 by Laszlo Boszormenyi, original patch by Reiner Herrmann.
• wise/2.4.1-19 by Sascha Steinbiss.
• xmltooling/1.6.0-2 by Ferenc W gner.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out yet. (Relevant changelogs did not mention reproducible builds.)

• comitup/0.5-1 by David Steele.
• dita-ot/1.5.3-2 by Mathieu Malaterre.
• dput/0.10.2 by Ben Finney.
• gnome-settings-daemon/3.21.90-2 by Michael Biebl.
• libkf5kipi/4:16.08.0-1 by Pino Toscano.
• libmpc/2:0.1~r475-2 by James Cowgill.
• taurus/4.0.1+dfsg-1 by Picca Fr d ric-Emmanuel.
• tcpwatch-httpproxy/1.3.1-2 by Toni Mueller.

The following 4 packages were not changed, but have become reproducible due to changes in their build-dependencies:

• djangorestframework
• roarplaylistd
• zope-maildrophost
• zope-replacesupport

Some uploads have addressed some reproducibility issues, but not all of them:

• amanda/1:3.3.9-1 by Jose M Calhariz.
• dispcalgui/3.1.6.0-2 by Christian Marillat, original patch by Chris Lamb.
• fastqtl/2.184+dfsg-4 by Dylan A ssi.
• grass/7.0.5~rc1-1~exp1 by Bas Couwenberg.
• kdevplatform/5.0-1 by Pino Toscano, original patch by Scarlett Clark.
• leveldb/1.19-1 by Alessio Treglia.
• libdevel-cover-perl/1.23-2 by gregor herrmann, original patch by Chris Lamb.
• linux/4.7.2-1 by Ben Hutchings, #830268 by Reiner Herrmann.
• lmms/1.1.3-4 by Javier Serrano.
• mayavi2/4.4.3-2.2 by Anton Gladky.
• opensaml2/2.6.0-2 by Ferenc W gner.
• perl/5.22.2-4 by Dominic Hargreaves, original patch and original patch by Chris Lamb.
• radare2/0.10.5+dfsg-1 by Sebastian Reichel, original patch by Chris Lamb.
• splint/3.1.2.dfsg1-3 by Rapha l Hertzog, original patch by Chris Lamb.

Patches submitted that have not made their way to the archive yet:

• #835673 filed against dacs by Chris Lamb.
• #835805 filed against dh-python by Chris Lamb.
• #835985 filed against nmh by Chris Lamb.
• #836609 filed against nostalgy by Chris Lamb.
• #835807 filed against pygtksourceview by Chris Lamb.
• #836004 filed against python-gflags by Chris Lamb.
• #835816 filed against rt-extension-customfieldsonupdate by Chris Lamb.
• #835834 filed against ruby-compass by Chris Lamb.
• #836605 filed against torque by Chris Lamb.
• #833176 filed against trafficserver by Reiner Herrmann.

Reviews of unreproducible packages 706 package reviews have been added, 22 have been updated and 16 have been removed in this week, adding to our knowledge about identified issues. 5 issue types have been added:

• timestamps_in_documentation_generated_by_ocamldoc
• timestamp_in_enc_files_added_by_texlive_fontinst
• cython_captures_build_path
• timestamps_in_perllocal_pod_manpages_generated_by_perl_extutils_mm_unix
• uname_output_in_python_debugging_symbols_caused_by_sysconfig_getplatform

1 issue type has been updated:

• timestamp_in_enc_files_added_by_texlive_fontinst

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (8)
• Lucas Nussbaum (3)

diffoscope development diffoscope development on the next version (60) continued in git, taking in contributions from:

• Mattia Rizzolo:
  • Better and more thorough testing
  • Improvements to packaging
  • Improvements to the ppu comparator

strip-nondeterminism development Mattia Rizzolo uploaded strip-nondeterminism 0.023-2~bpo8+1 to jessie-backports. A new version of strip-nondeterminism 0.024-1 was uploaded to unstable by Chris Lamb. It included contributions from:

• Chris Lamb:
  • Improve code quality of zip, jar, ar, png processors
• AYANOKOUZI, Ryuunosuke:
  • Preserve file attribute information of target file (#836075)

Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. disorderfs development Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. tests.reproducible-builds.org Debian: We now vary the GECOS records of the two build users. Thanks to Paul Wise for providing the patch. Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

Earlier today I was handling a reproducibility bug and decided I had to try a reproducibility test by myself. I tried reprotest, but I was being hit by a disorderfs issue and I was not sure whether the problem was with reprotest or not (at this point I cannot reproduce that anymore). So I decided to hack a simple script to that, and it works. I even included it in devscripts after writing a manpage. Of course reprotest is more complete, extensible, and supports arbitrary virtualization backends for doing the more dangerous/destructive variations (such as changing the hostname and other things that require root) but for quick tests debrepro does the job. Usage examples:

$ debrepro                                 # builds current directory
$ debrepro /path/to/sourcepackage          # builds package there
$ gbp-buildpackage --git-builder=debrepro  # can be used with vcs wrappers as well

debrepro will do two builds with a few variations between them, including $USER, $PATH, timezone, locale, umask, current time, and will even build under disorderfs if available. Build path variation is also performed because by definition the builds are done in different directories. If diffoscope is installed, it will be used for deep comparison of non-matching binaries. If you are interested and don t want to build devscripts from source or wait for the next release, you can just grab the script, save it as debrepro somewhere on your $PATH and make it executable.

What happened in the Reproducible Builds effort between Sunday August 14 and Saturday August 20 2016: Fasten your seatbelts Important note: we enabled build path variation for unstable now, so your package(s) might become unreproducible, while previously it was said to be reproducible given a specific build path it probably still is reproducible but read on for the details below in the tests.reproducible-builds.org section! As said many times: this is still research and we are working to make it reality. Media coverage Daniel Stender blogged about python packaging and explained some caveats regarding reproducible builds. Toolchain developments Thomas Schmitt uploaded xorriso which now obeys SOURCE_DATE_EPOCH. As stated in its man pages:

ENVIRONMENT
[...]
SOURCE_DATE_EPOCH  belongs to the specs of reproducible-builds.org.  It
is supposed to be either undefined or to contain a decimal number which
tells the seconds since january 1st 1970. If it contains a number, then
it is used as time value to set the  default  of  --modification-date=,
--gpt_disk_guid,  and  --set_all_file_dates.  Startup files and program
options can override the effect of SOURCE_DATE_EPOCH.

Packages reviewed and fixed, and bugs filed The following packages have become reproducible after being fixed:

• cadencii/3.3.9+svn20110818.r1732-5 by Ying-Chun Liu, original patch by Chris Lamb.
• cfengine3/3.7.4-3 by Antonio Radici.
• findbugs/3.1.0~preview2-1 by Emmanuel Bourg.
• gcpegg/5.1-14 by Bdale Garbee, original patch by Chris Lamb.
• gnunet-gtk/0.10.1-4 by Bertrand Marc, original patch by Chris Lamb.
• konwert/1.8-12 by Yann Dirson, original patch by Chris Lamb.
• link-grammar/5.3.8-2 by Jeremy Bicha, original patch by Chris Lamb.
• metastudent-data/2.0.1-1 by Sascha Steinbiss.
• pixmap/2.6pl4-20 by Paul Slootman, original patch by Chris Lamb Maria Valentina Marin a.k.a. Akira.
• python-afl/0.5.4-2 by Daniel Stender.
• tf5/5.0beta8-6 by Russ Allbery, original patch by Chris Lamb.
• triplane/1.0.8-2 by Markus Koschanyover, original patch by Chris Lamb.
• vips/8.3.3-1 by Laszlo Boszormenyi, original patch by Chris Lamb.
• xzgv/0.9.1-4 by Theodore Y. Ts'o, original patch by Chris Lamb.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• vulkan/1.0.21.0+dfsg1-1 by Timo Aaltonen.

The following 2 packages were not changed, but have become reproducible due to changes in their build-dependencies: tagsoup tclx8.4. Some uploads have addressed some reproducibility issues, but not all of them:

• cappuccino/0.5.1-4 by Breno Leitao, original patch by Chris Lamb.
• darkice/1.3-0.1 by Nicolas Bouleng.
• flask-restful/0.3.5-1 by Ond ej Nov , original patch by Chris Lamb.
• kodi/16.1+dfsg1-2 by Balint Reczey, original patch by Lukas Rechberger.
• libaws/3.3.2-3 by Nicolas Bouleng.
• liblog4ada/1.3-1 by Nicolas Bouleng.
• libxmlezout/1.06.1-8 by Nicolas Bouleng.
• ruby-rmagick/2.15.4+dfsg-2 by Antonio Terceir, original patch from Chris Lamb.
• taggrepper/0.05-2 by Kumar Appaiah, original patch and original patch from Maria Valentina Marin.
• xotcl/1.6.8-2 by Stefan Sobernig, original patch by Chris Lamb.

Patches submitted that have not made their way to the archive yet:

• #834755 against apt-cacher-ng by Chris Lamb.
• #834976 against auto-apt-proxy by Chris Lamb.
• #834549 against botan1.10 by Chris Lamb.
• #834861 against cookiecutter by Chris Lamb.
• #834779 against dicom3tools by Chris Lamb.
• #834859 against echoping by Chris Lamb.
• #834983 against eyed3 by Chris Lamb.
• #834735 against filepp by Chris Lamb.
• #834957 against flashrom by Chris Lamb.
• #834773 against gkrellm by Chris Lamb.
• #834292 against gr-radar by Chris Lamb.
• #834956 against ircd-irc2 by Chris Lamb.
• #834897 against jpy by Chris Lamb.
• #834452 against libdc0 by Chris Lamb.
• #834324 against libnss-ldap by Reiner Herrmann.
• #834945 against libquvi by Chris Lamb.
• #834534 against librep by Chris Lamb.
• #834537 against mozvoikko by Chris Lamb.
• #834857 against nagios-nrpe by Chris Lamb.
• #834316 against openocd by Reiner Herrmann.
• #834993 against oss4 by Reiner Herrmann.
• #834302 against pd-flite by Reiner Herrmann.
• #834754 against phpab by Chris Lamb.
• #834279 against phpldapadmin by Chris Lamb.
• #834277 against pybit by Chris Lamb.
• #834553 against pyicu by Chris Lamb.
• #834541 against ruby-pg by Chris Lamb.
• #835051 against sheepdog by Chris Lamb.
• #834896 against ttf-tiresias by Chris Lamb.
• #834780 against tuxpaint-config by Chris Lamb.
• #834988 against twitter-bootstrap3 by Chris Lamb.
• #834776 against uhub by Chris Lamb.
• #835061 against varnish by Chris Lamb.

Bug tracker house keeping:

• Chris Lamb pinged 164 bugs he filed more than 90 days ago which have a patch and had no maintainer reaction.

Reviews of unreproducible packages 55 package reviews have been added, 161 have been updated and 136 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated:

• Added user_in_documentation_generated_by_gsdoc
• Added locale_differences_in_pom

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (16)
• Santiago Vila (2)

diffoscope development Chris Lamb, Holger Levsen and Mattia Rizzolo worked on diffoscope this week. Improvements were made to SquashFS and JSON comparison, the https://try.diffoscope.org/ web service, documentation, packaging, and general code quality. diffoscope 57, 58, and 59 were uploaded to unstable by Chris Lamb. Versions 57 and 58 were both broken, so Holger set up a job on jenkins.debian.net to test diffoscope on each git commit. He also wrote a CONTRIBUTING document to help prevent this from happening in future. From these efforts, we were also able to learn that diffoscope is now reproducible even when built across multiple architectures:

< h01ger>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope.html shows these packages were built on amd64:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger>   and on i386:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger>   and on armhf:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb

And those also match the binaries uploaded by Chris in his diffoscope 59 binary upload to ftp.debian.org, yay! Eating our own dogfood and enjoying it! tests.reproducible-builds.org Debian related:

• show percentage of results in the last 24/48h (h01ger)
• switch python database backend to SQLAlchemy (Valerie)
• vary build path varitation for unstable and experimental for all architectures. (h01ger)

The last change probably will have an impact you will see: your package might become unreproducible in unstable and this will be shown on tracker.debian.org, while it will still be reproducible in testing. We've done this, because we think reproducible builds are possible with arbitrary build paths. But: we don't think those are a realistic goal for stretch, where we still recommend to use .buildinfo to record the build patch and then do rebuilds using that path. We are doing this, because besides doing theoretical groundwork we also have a practical goal: enable users to independently verify builds. And if they only can do this with a fixed path, so be it. For now To be clear: for Stretch we recommend that reproducible builds are done in the same build path as the "original" build. Finally, and just for our future references, when we enabled build path variation on Saturday, August 20th 2016, the numbers for unstable were:

suite	all	reproducible	unreproducible	ftbfs	depwait	not for this arch	blacklisted
unstable/amd64	24693	21794 (88.2%)	1753 (7.1%)	972 (3.9%)	65 (0.2%)	95 (0.3%)	10 (0.0%)
unstable/i386	24693	21182 (85.7%)	2349 (9.5%)	972 (3.9%)	76 (0.3%)	103 (0.4%)	10 (0.0%)
unstable/armhf	24693	20889 (84.6%)	2050 (8.3%)	1126 (4.5%)	199 (0.8%)	296 (1.1%)	129 (0.5%)

Misc. Ximin Luo updated our git setup scripts to make it easier for people to write proper descriptions for our repositories. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

One year ago (tomorrow) drat 0.1.0 was released. It held up rather well, but a number of small fixes and enhancements piled up, along with somewhat-finished to still-raw additions to the examples/ sections. With that, we are happy to announce drat release 0.1.1 which arrived on CRAN earlier today. drat stands for drat R Archive Template, and helps with easy-to-create and easy-to-use repositories for R packages. Since its inception in early 2015 it has found reasonably widespread adoption among R users because repositories is what we use. In other words, friends don't let friends use install_github(). Just kidding. Maybe. Or not. This version 0.1.1 builds on the previous release from one year ago. Several users sent in nicely focused pull request, and I added a bit of spit and polish here and there. The NEWS file (added belatedly in this release) summarises the release as follows:

Changes in drat version 0.1.1 (2016-08-07)

• Changes in drat functionality
  • Use dir.exists, leading to versioned Depends on R (>= 3.2.0)
  • Optionally pull remote before insert (Mark in PR #38)
  • Fix support for dots (Jan G. in PR #40)
  • Accept dots in package names (Antonio in PR #48)
  • Switch to htpps URLs at GitHub (Colin in PR #50)
  • Support additional fields in PACKAGE file (Jan G. in PR #54)
• Changes in drat documentation
  • Further improvements and clarifications to vignettes
  • Travis script switched to run.sh from our fork
  • This NEWS file was (belatedly) added

Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

As of yesterday, I am the new maintainer of pristine-tar. As it is the case for most of Joey Hess creations, it is an extremely useful tool, and used in a very large number of Debian packages which are maintained in git. My first upload was most of a terrain recognition nature: I did some housekeeping tasks, such as making the build idempotent and making sure all binaries are built with security hardening flags, and wrote a few automated test cases to serve as build-time and run-time regression test suite. No functional changes have been made. As Joey explained when he orphaned it, there are a few technical challenges involved in making sure pristine-tar stays useful in the future. Although I did read some of the code, I am not particularly familiar with the internals yet, and will be more than happy to get co-maintainers. If you are interested, please get in touch. The source git repository is right there.

What happened in the reproducible builds effort between February 28th and March 5th:

Toolchain fixes

• Antonio Terceiro uploaded gem2deb/0.27 that forces generated gemspecs to use the date from debian/changelog.
• Antonio Terceiro uploaded gem2deb/0.28 that forces generated gemspecs to have their contains file lists sorted.
• Robert Luberda uploaded ispell/3.4.00-5 which make builds of hashes reproducible.
• C dric Boutillier uploaded ruby-ronn/0.7.3-4 which will make the output locale agnostic. Original patch by Chris Lamb.
• Markus Koschany uploaded spring/101.0+dfsg-1. Fixed by Alexandre Detiste.

Ximin Luo resubmitted the patch adding the --clamp-mtime option to Tar on Savannah's bug tracker. Lunar rebased our experimental dpkg on top of the current master branch. Changes in the test infrastructure are required before uploading a new version to our experimental repository. Reiner Herrmann rebased our custom texlive-bin against the latest uploaded version.

Packages fixed The following 77 packages have become reproducible due to changes in their build dependencies: asciidoctor, atig, fuel-astute, jekyll, libphone-ui-shr, linkchecker, maven-plugin-testing, node-iscroll, origami-pdf, plexus-digest, pry, python-avro, python-odf, rails, ruby-actionpack-xml-parser, ruby-active-model-serializers, ruby-activerecord-session-store, ruby-api-pagination, ruby-babosa, ruby-carrierwave, ruby-classifier-reborn, ruby-compass, ruby-concurrent, ruby-configurate, ruby-crack, ruby-css-parser, ruby-cucumber-rails, ruby-delorean, ruby-encryptor, ruby-fakeweb, ruby-flexmock, ruby-fog-vsphere, ruby-gemojione, ruby-git, ruby-grack, ruby-htmlentities, ruby-jekyll-feed, ruby-json-schema, ruby-listen, ruby-markerb, ruby-mathml, ruby-mini-magick, ruby-net-telnet, ruby-omniauth-azure-oauth2, ruby-omniauth-saml, ruby-org, ruby-origin, ruby-prawn, ruby-pygments.rb, ruby-raemon, ruby-rails-deprecated-sanitizer, ruby-raindrops, ruby-rbpdf, ruby-rbvmomi, ruby-recaptcha, ruby-ref, ruby-responders, ruby-rjb, ruby-rspec-rails, ruby-rspec, ruby-rufus-scheduler, ruby-sass-rails, ruby-sass, ruby-sentry-raven, ruby-sequel-pg, ruby-sequel, ruby-settingslogic, ruby-shoulda-matchers, ruby-slack-notifier, ruby-symboltable, ruby-timers, ruby-zip, ticgit, tmuxinator, vagrant, wagon, yard. The following packages became reproducible after getting fixed:

• air-quality-sensor/0.1.4-1 uploaded by Benedikt Wildenhain, fixed upstream, original patch by Chris Lamb.
• device3dfx/2013.08.08-4 by Guillem Jover.
• fldigi/3.23.08-1 by Kamal Mostafa.
• fltk1.1/1.1.10-22 by Aaron M. Ucko.
• freeimage/3.17.0+ds1-2 by Ghislain Antony Vaillant.
• gimagereader/3.1.2+git368fa8f-2 by Philip Rinn.
• ginkgocadx/3.7.5-1 by Gert Wollny, fixed upstream.
• jadetex/3.13-17 by Norbert Preining.
• opensips/2.1.2-1 by Razvan Crainea.
• ruby-sqlite3/1.3.11-2 uploaded by C dric Boutillier, original patch by Lunar.
• runawk/1.6.0-2 uploaded by Andrew Shadura, patch by Reiner Herrmann.
• systraq/20160303-1 by Joost van Baal-Ili .

Some uploads fixed some reproducibility issues, but not all of them:

• auto-multiple-choice/1.2.1-4 by Georges Khaznadar.
• avfs/1.0.3-1 uploaded by Michael Meskes, original patch by Chris Lamb.
• console-setup/1.138 uploaded by Anton Zinoviev, original patch by Reiner Herrmann.
• gromacs/5.1.2-1 by Nicholas Breen.
• mrrescue/1.02c-2 by Alexandre Detiste.
• usb-modeswitch-data/20160112-2 by Didier Raboud.

Patches submitted which have not made their way to the archive yet:

• #816209 on elog by Reiner Herrmann: use printf instead of echo which is shell-independent.
• #816214 on python-pip by Reiner Herrmann: removes timestamp from generated Python scripts.
• #816230 on rows by Reiner Herrmann: tell grep to always treat the input as text.
• #816232 on eficas by Reiner Herrmann: use printf instead of echo which is shell-independent.

Florent Daigniere and bancfc reported that linux-grsec was currently built with GRKERNSEC_RANDSTRUCT which will prevent reproducible builds with the current packaging.

tests.reproducible-builds.org pbuilder has been updated to the last version to be able to support Build-Depends-Arch and Build-Conflicts-Arch. (Mattia Rizzolo, h01ger) New package sets have been added for Subgraph OS, which is based on Debian Stretch: packages and build dependencies. (h01ger) Two new armhf build nodes have been added (thanks Vagrant Cascadian) and integrated in our Jenkins setup with 8 new armhf builder jobs. (h01ger)

strip-nondeterminism development strip-nondeterminism version 0.016-1 was released on Sunday 28th. It will now normalize the POT-Creation-Date field in GNU Gettext .mo files. (Reiner Herrmann) Several improvements to the packages metadata have also been made. (h01ger, Ben Finney)

Package reviews 185 reviews have been removed, 91 added and 33 updated in the previous week. New issue: fileorder_in_gemspec_files_list. 43 FTBFS bugs were reported by Chris Lamb, Martin Michlmayr, and gregor herrmann.

Misc. After merging the patch from Dhiru Kholia adding support for SOURCE_DATE_EPOCH in rpm, Florian Festi opened a discussion on the rpm-ecosystem mailing list about reproducible builds. On March 4th, Lunar gave an overview of the general reproducible builds effort at the Internet Freedom Festival in Valencia.

Earlier today I was made aware by Holger of the results of our reproducibility efforts during the sprint. I would like to thank Lunar for pinging us about the issue, and Holger for pointing me to updated results. The figure below depicts a stacked area chart where the X axis is time and the green area is reproducible packages. Red is packages that fail to build, and Orange are unreproducible packages I was able to book accommodation for the sprint attendees very close to both my place and the sprint venue, what was very useful but also had this downside of them not being able to see much of city. As the final day of the sprint was getting closer, we decided to have a different lunch to allow them to see one of the most famous local landmarks, the botanical gardens. So we headed down to the botanical gardens, grabbed a few items for lunch at the park coffee shop, and set out to visit this very beautiful place. I have to say that there is the place were I usually take every visitor I have. We were joined by Gioavani who had just arrived for the the MiniDebconf on the following weekend. The final lists of accomplishments of the day was again very impressive

• r10k 2.1.1-2
• run massive update on team repositories
  • bump Standards-Version
  • fix Vcs-* fields
  • drop version in gem2deb build-dependency
  • set debhelper compatibility level to 9
  • update the default ruby-tests.rake
• day 4 report
• uploaded ruby-faraday-middleware-multi-json 0.0.6-2
• uploaded ruby-powerpack 0.1.1-2
• uploaded ruby-contracts 0.13.0-1
• uploaded ruby-chef-config 12.7.2-1 (NEW)
• uploaded ruby-foreigner #808530 and asked it removal from the NEW queue (was already ROMed)
• filled for RM ruby-opengraph-parser (#816752)
• new how-can-i-help version developed and uploaded
• uploaded ruby-romkan to unstable (from exp)
• uploaded ruby-rinku to unstable (from exp)
• uploaded ruby-ole to unstable (from exp)
• uploaded ruby-net-ldap to unstable (from exp)
• uploaded ruby-rack-mobile-detect to unstable (from exp)
• uploaded gem2deb 0.28 to help with reproducible builds: filenames are now sorted
• uploaded rails 2:4.2.5.2-2 with packaging improvements
  • run unit tests during the build and on CI
  • apply upstream patch to fix ActiveRecord breakage under Ruby 2.3
• pushed a ton of tags for existing uploads
• merged improvements to the team master repository
  • review/cleanup the contents of the repository
  • improved helper scripts to automate the workflow (upload, build, new-upstream, etc)
• followed up on ruby2.3 transition, filed #816698 against subversion because of ftbfs on mips, mipsel
• put ruby-cocoon into a better state
• uploaded ruby-plist
• gem2deb: gem2tgz will now create foo.gemspec (easier to patch) instead of metadata.yml
• gemwatch: ditto
• close #794139 jekyll bug (unreproducible)
• close #798934 ruby-ffi-rzmq bug (unreproducible)
• closed ftbfs #816586 #800057 #784699 as unreproducible
• reassigned #760952 #680297 to ruby2.3 (from ruby2.2)
• investigated how to list packages with non-buildd-binary uploads
• ScottK has removed ruby2.1 from unstable!

By the end of the afternoon I asked everyone to fill out a simple retrospective list, what we can use later to make future sprints better and better. Below are the results we got. What was good:

• restricted room hours actually made for a nice rhythm (did not apply for a long time )
• very good food
• very cheap food!
• longer period makes the effort of travel more worthwhile
• many participants and longer sprint than usual allowing more work to be done
• good preparation with clear goals, make the sprint usefull
• patience with the less experienced participants
• RFS very fast
• Ant nio is an excellent host
• You all are so helpful
• great dinners

What could be better:

• room too close to the street, too much vehicle noise, but sometimes nice music
• more coffee ^W meat
• could know more portugues so ordering food would have been easier
• debian infra could have not been down during the sprint

The night ended at Bar do Alem o ( The German s Bar ). Both their beer and their food are very good, but I don t have enough elements to vouch for their authenticity. :) We were joined by Giovani (who we also met earlier in the botanic gardens), and by Paulo and Daniel who are organizing the MiniDebconf. And that is the end of this year s Debian Ruby team sprint. I hope we do it all over again next year.

What happened in the reproducible builds effort between February 21th and February 27th:

Toolchain fixes Didier Raboud uploaded pyppd/1.0.2-4 which makes PPD generation deterministic. Emmanuel Bourg uploaded plexus-maven-plugin/1.3.8-10 which sorts the components in the components.xml files generated by the plugin. Guillem Jover has implemented stable ordering for members of the control archives in .debs. Chris Lamb submitted another patch to improve reproducibility of files generated by cython.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dctrl-tools, debian-edu, dvdwizard, dymo-cups-drivers, ekg2, epson-inkjet-printer-escpr, expeyes, fades, foomatic-db, galternatives, gnuradio, gpodder, gutenprint icewm, invesalius, jodconverter-cli latex-mk, libiio, libimobiledevice, libmcrypt, libopendbx, lives, lttnganalyses, m2300w, microdc2, navit, po4a, ptouch-driver, pxljr, tasksel, tilda, vdr-plugin-infosatepg, xaos. The following packages became reproducible after getting fixed:

• afterstep/2.2.12-7 by Robert Luberda.
• arachne-pnr/0~20150927gitefdb026-2 uploaded by Ruben Undheim, patch by Dhole.
• astroquery/0.3.1+dfsg-2 by Vincent Prat.
• compton-conf/0.1.0+20151226-2 uploaded by Alf Gaida, original patch by Dhole.
• disque/1.0~rc1-5 uploaded by Chris Lamb, issue identified by Reiner Herrmann.
• foo2zjs/20151024dfsg0-2 by Didier Raboud.
• gnugo/3.8-9 uploaded by Martin A. Godisch, original patch by Reiner Herrmann.
• hplip/3.16.2+repack0-4 by Didier Raboud.
• ibus-braille/0.1.2.99+git1.a95477d-4 by Samuel Thibault.
• iputils/3:20150815-1 by Noah Meyerhans, original patch by Juan Picca.
• jimtcl/0.76-2 by Didier Raboud.
• jodconverter/2.2.2-8 uploaded by Samuel Thibault, original patch by Reiner Herrmann.
• jts/1.14+ds-1~exp1 by Bas Couwenberg.
• loadlin/1.6f-5 by Samuel Thibault.
• lximage-qt/0.4.0+20160108-3 by ChangZhuo Chen ( ), patch by Dhole.
• modello/1.8.3-2 by Emmanuel Bourg.
• obconf-qt/0.9.0+20151227-2 uploaded by Alf Gaida, original patch by Dhole.
• pcmanfm-qt/0.10.1-2 uploaded by Alf Gaida, original patch by Dhole.
• pnm2ppa/1.13-7 by Didier Raboud.
• screengrab/1.95+20160128-2 uploaded by Alf Gaida, original patch by Dhole.
• sip4/4.17+dfsg-2 uploaded by Scott Kitterman, original patch by Reiner Herrmann.
• spades/3.7.0+dfsg-1 by Sascha Steinbiss.
• sphinxtrain/1.0.8+5prealpha-4 by Samuel Thibault.
• tcsh/6.18.01-5 uploaded by Thomas Lange, original patch by Reiner Herrmann.
• ubertooth/2015.09.R2-4 by Ruben Undheim.
• watchdog/5.15-1 by Michael Meskes.
• xfonts-a12k12/1-12 uploaded by Nobuhiro Iwamatsu, original patch by Chris Lamb.

Some uploads fixed some reproducibility issues, but not all of them:

• gridsite/2.2.6-2 by Mattias Ellert.
• gsoap/2.8.28-2 by Mattias Ellert.
• natbraille/2.0rc3-3 by Samuel Thibault.
• pairs/4:15.04.3-1 uploaded by Maximiliano Curia, original patch by Scarlett Clark.

tests.reproducible-builds.org The reproducibly tests for Debian now vary the provider of /bin/sh between bash and dash. (Reiner Herrmann)

diffoscope development diffoscope version 50 was released on February 27th. It adds a new comparator for PostScript files, makes the directory tests pass on slower hardware, and line ordering variations in .deb md5sums files will not be hidden anymore. Version 51 uploaded the next day re-added test data missing from the previous tarball. diffoscope is looking for a new primary maintainer.

Package reviews 87 reviews have been removed, 61 added and 43 updated in the previous week. New issues: captures_shell_variable_in_autofoo_script, varying_ordering_in_data_tar_gz_or_control_tar_gz. 30 new FTBFS have been reported by Chris Lamb, Antonio Terceiro, Aaron M. Ucko, Michael Tautschnig, and Tobias Frost.

Misc. The release team reported on their discussion about the topic of rebuilding all of Stretch to make it self-contained (in respect to reproducibility). Christian Boltz is hoping someone could talk about reproducible builds at the openSUSE conference happening June 22nd-26th in N rnberg, Germany.

As the day 4 of the Debian Ruby team sprint in Curitiba unfolded, we have now fixed a total of more than 70 build failure bugs, managed to almost finish the Ruby 2.3 transition to be good to migrate into testing, and bootstrapped some documentation that will help new contributors get up to speed with Ruby packaging faster. We have also requested the removal of several packages that are either severely outdated, abandoned upstream, beyond repair, utterly wrong, or in some cases, all of the above. The full list of work items finished yesterday is:

• filed for RM ruby-rubysl-test-unit (no rdeps, duplicates ruby-test-unit)
• filed for RM ruby-literati (no rdeps left)
• uploaded ruby-certificate-authority 0.1.6-2
• uploaded ruby-pdf-reader 1.4 (Closes: FTBFS #795763)
• uploaded ruby-http 1.0.2-1 (Closes: #795752)
• raise severity of FTBFS bugs with ruby2.3
• day 3 report
• NMU pcs and upload to DELAYED/2 to remove dependency on ruby-monkey-lib and build-dependency on ruby2.1-dev
• upload ruby-parslet (Closes: #795046)
• initial documentation of packaging workflow with updated helper scripts in the team repo https://wiki.debian.org/Teams/Ruby/Workflow
• ask for the removal of ruby-rspec-longrun
• ruby-opengraph-parser upstream bug; upstream unresponsive; asked uploader about removal (no rdeps)
• fix ruby-celluloid-io for ruby2.3
• fixed ruby-buff-extension to work on ruby2.3
• uploaded newer ruby-varia-model upstream to work with newer ruby-hashie
• ask for the removal of mdpress
• uploaded ruby-buff-config, ruby-semverse, ruby-buff-ruby-engine, ruby-buff-ignore, ruby-buff-shell-out to remove spork
• filed for RM spork (obsolete, broken)
• filed for RM ruby-gsl on failing archs
• upload ruby-solve (Closes: #816359, thanks zeha!)
• update ruby-standalone to work properly with ruby2.3 (needs to remove the rake binary)
• upload ruby-memfs
• ruby-rack (1.6.4-3) (ROM ruby-memcache-client)
• ruby-parslet 1.7.1-1 #795046
• upload ruby-listen
• upload ruby-clockwork
• upload ruby-bio with a patch to avoid transient FTBFS because of undefined class names
• fixed ruby-fakeweb ftbfs
• cleaned up old repositories on git.d.o
• investigated PET status (appears to somewhat work, but mostly decaying)
• uploaded ruby-ogginfo
• upload ruby-libxml (agan!)
• upload racc 1.4.14-1
• upload ruby-sidekiq-cron
• how-can-i-help updated
• racc 1.4.14-1 (not ruby-racc ;-))

We also managed to flirt with 2 capital sins. For those who care about these things, which I don t (but I still care about you), I guess 2 out of 7 still means we are good? :-) I few people that I will not name complained that they hadn t had enough steak on the previous night, so we set out to visit a traditional all-you-can-eat Brazilian steakhouse ( churrascaria ). I made a reservation at Jardins Grill and there you have gluttony. I am pretty sure that not enough steak wasn t an issue last night. You can see how happy, despite being full to almost the point of being sick, everyone was. A disjunct set of people, who I will also not name, were very disappointed to find out that the ruby-tinder package has absolutely nothing to do with Tinder but were still very active on the later. Maybe Friday night we will have to split the group into a lust-free family party and a Tinder party.

Day 3 was again a full of useful work. Since the beginning of the sprint, we were able to fix more than 50 FTBFS bugs, alongside general quality improvements in the packages.

in the Debian jargon, FTBFS means that a package fails to build from source , which in Debian is a critical bug because users need to be able to produce binary packages from their source code to fully exercise the free software principles.

An important milestone that was also achieved on day 3 was the upload of ruby-defaults 1:2.3.0+1, making ruby2.3 the new default version of Ruby. That is the version that will shipped in the next Debian release, codenamed stretch. This is the culmination of a joint effort between the Ruby team and Debian Release Team that involves rebuilding a little more than 130 packages that use the Ruby C API to make sure everything will just work on upgrades, both from the previous stable release, and from earlier snapshots of the current development release. Another small change that will have a big impact for Debian and for free software was an improvement to gem2deb that fixes a reproducibility issue in Ruby packages and will help currently more than 100 Ruby packages become reproducible. The full list of items that have been worked on is this:

• sponsor ruby-github-linguist. Closes: #815908. Thanks balasankar
• rebuild remaining failing packages 55 remaining!
• uploaded ruby-em-synchrony (FTBFS #800029)
• day 2 report blog post
• ruby-certificate-authority-0.1.6-1 (#816506)
• filed for ruby-grack removal
• uploaded ruby-em-synchrony
• uploaded ruby-mathml (FTBS #800029)
• uploaded new upstream version of ruby-redis which claims better ruby2.2 support
• uploaded pry (0.10.3-2), hoping it will build correctly on the buildd
• uploaded ruby-hashery
• rails security issues: CVE-2016-2098, CVE-2016-2097
  • fixed in unstable on 2:4.2.5.2-1
  • contacted security team for jessie update
• uploaded ruby-ruby-engine, ruby-ruby-version, ruby-rspec-pending-for to NEW
• updated ruby-hashie to new upstream version
• ruby-remcached (fix FTBFS)
• made the switch to ruby2.3 in unstable *********
• filed https://bugs.ruby-lang.org/issues/12137 as ruby-awesome-print triggers this
• uploaded ruby-rspec 3.4
• uploaded ruby-rspec-rails 3.4
• uploaded ruby-grack 2.0.2-2 (fix FTBFS)
• ruby-memcache-client (#797256) => RoM ruby-memcache-client (#816538) + https://github.com/rack/rack/issues/1025
• upload ruby-acts-as-lists (Closes: FTBS #816482)
• ruby-beautify: fixed and uploaded
• filed for ruby-foreigner removal (+ fixed diaspora dependency)
• uploaded newer ruby-origin to fix FTBFS
• uploaded ruby-responders
• gem2deb hacking
  • dh-make-ruby will not overwrite debian/copyright anymore
  • force generated gemspecs to use date from debian/changelog. fixes https://tests.reproducible-builds.org/issues/unstable/timestamps_in_gemspec_files_issue.html
  • uploaded version 0.27 to unstable
• make ruby-riddle tests run and work during build
• ruby-flexmock 2.0.4-1 -> uploaded
• RoM ruby-memcache-client (done #816538)
• upload nanoc 4.1.4 and fix tests for ruby2.3
• Upload new upstream version of ruby-test-after-commit
• uploaded newer upstream for ruby-origin fixing FTBFS bug

The day ended at Outback, where we had an amount of beer that led us to formulate what we will now call the One-Sided Dinner Booth Problem. In a party arranged like above, when the people closest to wall need to go alleviate themselves of some beer, you basically have to perform a removal from the bottom of a stack, which requires popping all the elements at the top. When they come back, you have to options:

• place them back at their original position, in which case
  • the people who stayed have to either stay standing or get up all over again (not sure what is worse)
  • the odds are that the next person to leave will require less effort from the others
• shift everyone to places against the wall and place the one coming back at the very end of the bench away from the wall, in which case
  • the people who stayed are perhaps less annoyed, but you have to rearrange plates and pints
  • it is less likely that the next time will require less effort from the others

The One-Sided Dinner Booth Problem is finding the optimal data structure and algorithm for this situation. It is postulated that this is an NP-complete problem, and that only probabilistic solutions are cost-effective.

Day 1 ended with dinner at a Yamato, my preferred Japanese restaurant in the city. Curitiba has a very large Japanese community, and lots of Japanese restaurants. Yamato, however, is the only one were you will stumble upon senior Japanese people, probably first or second generation immigrants, what I guess says something about its authenticity. Right after breaking for lunch, but before actually going out, we made what so far is official group photo (I might try again as the shot was not a really good one). Of course the most interesting part was the actual work that was done, and day 2 list is not less impressive than the day before:

• ruby-albino 1.3.3-4 (#813644) @sbadia
• #816256: ruby-versionomy: FTBFS: require : cannot load such file blockenspiel/unmixer_mri (LoadError)
• investiage ruby2.1 removal
• upload ruby-beautify
• upload ruby-aws-sdk
• made ruby-blockenspiel arch:all again
• filed RC bug against pcs depending on ruby2.1-dev
• ruby-dev in experimental now supports pkg-config ruby so packages can use pkg-config to build against the current default Ruby
• ruby-libxml (some tests skipped)
• fixed FTBFS #804794 in subtle delaying ruby2.1 removal (now in DELAYED/2)
• mailed debian-ruby and ask people to test ruby2.3 as default
• #816162: ruby-zentest: FTBFS: Failure: TestZenTest#test_testcase9
• ruby-sshkit 1.9.0~rc1-1 (795118)
  • capistrano 3.4.0-1 + ruby-sshkit 1.9.0~rc1-1 => http://paste.debian.net/plain/410699
• filed for ruby-patron removal
• updated and uploaded ruby-pygments.rb updated + patch for tests (fix #812940)
• discussed dependency cycle resolution
  • will keep cycle between ruby2.3 and rake (as they really need each other)
  • will break cycle from pure ruby packages to ruby when they are depended on by ruby2.3
  • for same package set, enable Multi-Arch: foreign when needed
• uploaded ruby-power-assert (M-A foreign, break ruby cycle)
• uploaded ruby-did-you-mean (M-A foreign, break ruby cycle)
• uploaded ruby-minitest (M-A foreign, break ruby cycle)
• uploaded ruby-test-unit (M-A foreign, break ruby cycle)
• uploaded rake (M-A foreign)
• uploaded ruby-net-telnet (M-A foreign)
• discussed removal of libruby metapackage, filed bug against dh-make-drupal (only rdep) #816417
• uploaded yard (Closes: #812809)
• rerun failed builds
• closed #798539 (not applicable on current version of ruby-webmock)
• closed #816321 fixed by ruby2.3 upload
• uploaded ruby-cliver (by @lucasmoura)
• uploaded ruby-celluloid-supervision ( closes: #810770)
• uploaded ruby-celluloid-fsm ( closes: #815107)
• uploaded ruby-raindrops
• investigated status of dhelp, ruby-mysql again
• ruby-http-form-data (1.0.1+gemwatch-1) @sbadia
• nmu ohcount (DELAYED/2)
• updated trocla to newest upstream, fixing #816257
• reassign #816358 from ruby-safe-yaml to ruby2.3
• close #816120 (transient bug in ruby-default-value-for, waiting for ruby-sqlite3 binNMU)
• upload updated ruby-rc4
• uploaded new upstream version of pry, fixing FTBFS with ruby2.3
• ruby-rb-inotify , required to new ruby-listen( need ruby-rb-inotify >= 0.9.7)
• applied patch from upstream to ruby2.3 for #816358
• uploaded new upstream version for coderay supporting ruby2.3

On Monday C dric told us that he and Sebastien had brought a bottle of French wine and some smelly French cheeses, and suggested that in the best Debian tradition we should have a Mini Cheese and Wine Party . Sure thing! Luckily there is a farmer s market 2 blocks from home on Tuesdays mornings, where I usually buy my fruits, vegetables, and cheese & friends, so the timing was perfect. I went shopping early in the morning, and bought a few things, and was back before it was the time to go to UTFPR. After the day-long hacking session we stopped by another store nearby to buy a few extra bottles of wine and other snacks. At night, in my place, I ended up playing cheese master. There was enough food that at the end we were all very full. And with the spokesperson task of the day done, off to hacking I am!

This year s Debian Ruby team sprint started today here at Curitiba. Everyone arrived fine, and we started working at the meeting room we have booked for the week at Curitiba campus of the Federal Technical University of Paran . The room is at the Department of Business and Community Relations , what makes a lot of sense! :-) The day started with a quick setup, with a simple 8-port switch, and a couple of power strips. It tooks us a few minutes to figure what was blocked or not on the corporate network, and almost everyone who needs connections that are usually blocked in such environments already had their VPN setups so we were able to get started right after that. We are taking notes live on mozilla s piblic etherpad site Today we accomplished quite a lot:

• analyzed the pending issues for the ruby2.3 transition issues, categorizing the missing packages into needs a binNMU now , needs a binNMU after switching the default to ruby2.3 , and broken .
• ruby-defaults uploaded to experimental switching to ruby2.3 as default, and dropping support for ruby2.2
• ruby-gsl fixed to build against GSL 2.x (was blocking ruby2.3 transition)
• #816253: ruby-fast-gettext: fix FTBFS issue and import a new upstream
• ruby-aws: several issues fixed
• ruby-binding-of-caller: fixed rubygems-integration
• fixed ruby-kakasi-ffi (again)
• made ruby-blockenspiel arch:any again
• ruby-fast-gettext 1.0.0-1 (fix ftbfs 816253)
• ruby-aws-sdk: new upstream, debcheck fix and several bumps
• ruby-fcgi dropped transitional packages + refreshed packaging with -w
• ruby-sshkit 1.8.0
• ruby-beautify: new upstream and few minor fixes
• asciidoctor (new version sponsored)
• capistrano 3.4.0 (ftbfs #795724, #802734)
• #816254: ruby-packetfu: FTBFS: expected NameError with uninitialized constant PacketFu::FakePacket , got #
• updated rake to 10.5.0; making it not include -I/usr/lib/ruby/vendor_ruby when running tests
• triaged an closed open bugs on ruby-httpclient that do not apply anymore.
• updated ruby-httpclient to get rid of warnings in apt-listbugs under Ruby 2.3
• ruby-packetfu (fix ftbfs #816254) by @kanashiro
• investigated extension/rdepends build failing with ruby2.3-only
• applied upstream patch to ruby to fix extension ftbfs when extension uses c++
• ruby-albino 1.3.3-4 (#813644)
• basic user-level testing using ruby2.3 as default:
  • chef WORKS mostly; it seems ohai segfaults some times
  • rails autopkgtest FIXED Could not find gem binding_of_caller (>= 0.7.2) , which is required by gem web-console (~> 2.0) , in any of the sources.
  • vagrant WORKS
  • redmine autopkgtest WORKS
  • apt-listbugs FIXED warnings (e.g. try apt-listbugs list $pkg ); caused by ruby-httpclient
  • nanoc WORKS
• capistrano 3.4.0-1 (#795724, #802734) by @sbadia + @terceiro

It s been almost 2 years since the Debian Continuous Integration project has been launched, and it has proven to be a useful resource for the development of Debian. I have previously made a an introductory post, and this this is an update on the latest developments. Infrastructure upgrade Back in early 2014 when Debian CI was launched, there were less than 200 source packages with declared test suite metadata, and using a single worker machine polling the archive for updates and running tests sequentially in an infinite loop ( the simplest thing that could possibly work ) was OK-ish. Then our community started an incredible, slow and persistent effort to prepare source packages for automated testing, and we now have almost 5,000 of them. The original, over-simplistic design had to be replaced. The effort of transforming debci in a distributed system was started by Martin Pitt, who did an huge amount of work. In the latest months I was able to complete that work, to a point where I am confident in letting it run (mostly) unatended. We also had lots of contributions to the web UI from Brandon Fairchild, who was a GSOC intern in 2014, and continues to contribute to this date. All this work culminated in the migration from a single-worker model to a master/workers setup, currently with 10 worker nodes. On busy periods all of those worker nodes will go on for days with full utilization, but even then the turnaround between package upload and a test run is now a lot faster than it used to. Debian members can inspect the resource usage on those systems, as well as the length of the processing queue, by browsing to the corresponding munin instance (requires authentication via a SSL client certificated issued by sso.debian.org). The system is currenly being hosted on a Amazon EC2 account sponsored by Amazon. The setup is fully automated and reproducible. It is not fully (or at all) documented yet, but those interested should feel free to get in touch on IRC (OFTC, #debci) Testing backend changed from schroot to lxc Together with the infrastructure updates, we also switched to using lxc instead of schroot as backend. Most test suites should not be affected by this, but the default lxc settings might cause some very specific issues in a few packages. See for example #806542 ( liblinux-prctl-perl: autopkgtest failures: seccomp, capbset ) Adding support for KVM is also in the plans, and we will get to that at some point. Learn more If you want to learn more on how you can add tests for your package, a good first start is the debci online documentation (which is also available locally if you install debci ). You might also be interested in watching the live tutorial (WebM, 469 MB!) that has been presented at Debconf 15 earlier this year, full of tips and real examples from the archive. It would be awesome if someone wanted to transcribe that into a text tutorial ;-) How to get involved There are a few ways you can contribute: autodep8. if you are knowledgeable on a subset of packages that are very similar and can have their tests executed in a similar way, such as $Language libraries , you might consider writing a test metadata generator so that each package does not need to declare a debian/tests/control file explicitly, requiring only The Testsuite: header in debian/control. Ruby and Perl are already covered, and there is initial support for NodeJS. Adding support for new types of packages is very easy. See the source repository. If you manage to add support for your favorite language, please get in touch so we can discuss whitelisting the relavant packages in ci.debian.net so that they will get their tests executed even before being uploaded with the proper Testsuite: control field. autopkgtest. autopkgtest is responsible for actually running your tests, and you can use it to reproduce test runs locally. debci. debci is the system running in ci.debian.net (version 1.0, currently in testing, is exactly what is running up there, minus a version number and a changelog entry). It can also be used to have private clones of ci.debian.net, e.g. for derivatives or internal Debian-related development. See for example the Ubuntu autopkgtest site. Getting in touch For maintainer queries and general discussion:

• mailing list: debian-qa@lists.debian.org
• IRC: #debian-qa on OFTC. Feel free to highlight terceiro

For the development of debci/autopkgtest/autodep8

• mailing list: autopkgtest-devel@lists.alioth.debian.org
• IRC: #debci on OFTC